Skip to content

gh-161977: Fix quadratic time in textwrap when breaking a long word - #161978

Open
thejesh23 wants to merge 8 commits into
python:mainfrom
thejesh23:textwrap-long-word
Open

thejesh23 wants to merge 8 commits into
python:mainfrom
thejesh23:textwrap-long-word

Conversation

@thejesh23

@thejesh23 thejesh23 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

textwrap.wrap() took quadratic time on a long word: each line broke off one piece and copied the rest of the word.

When a line after the first starts with a long word, _wrap_chunks() now splits it into line-width pieces at once, since all those lines have the same width. Words with whitespace, or with hyphens when break_on_hyphens is true, keep the old path. The output is unchanged.

input main this PR improvement
base64 of 1 MB, wrap(s, 76) 0.23 s 0.06 s 74% less time (3.8x)
base64 of 4 MB, wrap(s, 76) 3.00 s 0.25 s 92% less time (12x)
base64 of 8 MB, wrap(s, 76) 11.23 s 0.51 s 95% less time (22x)
'x' * 3_200_000, wrap(s, 70) 1.10 s 0.14 s 87% less time (7.8x)

Testing: a new test in test_textwrap, identical output from old and new code on 260,000 random inputs, and the full test suite passes.

AI disclosure: prepared with the help of Claude Code; I reviewed the change and the test results.

TextWrapper._handle_long_word() broke off one line's worth of a long
word at a time and stored the rest back, copying the rest of the word
for every line.  Break the rest of the word into the pieces of all the
following lines in one pass instead.  They all have the same width, so
the result is the same.
Comment thread Lib/textwrap.py Resolved
Comment thread Lib/textwrap.py Resolved
@thejesh23

Copy link
Copy Markdown
Contributor Author

@maurycy thanks for the review I'm not sure why you have deleted your posted comment actually its a right catch , NEWS entry: use :mod: instead of naming functions.

@maurycy

maurycy commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

@thejesh23 I'm sorry, I've removed the one about NEWS accidentially, thank you for addressing it in 43c0f59.

That said:

  • Did you use PGO+LTO build to benchmark?
  • I'd move to the _wrap_chunks(). I'm not sure if the current complexity is worth it, especially given that 10-100k+ words (where this change shines) aren't really that frequent.
  • Could you not use AI to write text? Generally: https://devguide.python.org/getting-started/ai-tools/ but it's just hard to read

Thank you! 💚

@thejesh23

thejesh23 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor Author

@maurycy no worries .

  • Did you use PGO+LTO build to benchmark?

Actually used normal optimised build (./configure with -O3), without --enable-optimizations or --with-lto

  • I'd move to the _wrap_chunks(). I'm not sure if the current complexity is worth it, especially given that 10-100k+ words (where this change shines) aren't really that frequent.

Agree to move the splitting into _wrap_chunks().
Wrapping base64 data, a common idiom for PEM keys and MIME bodies, is one long word with no spaces or hyphens. I measured wrap(b64, 76) just now.

Still getting used to the used to all of repo rules . Irrespective of repo rules for responding/acting on comments I don't use AI ::)

Comment thread Lib/textwrap.py Resolved
@bedevere-app bedevere-app Bot added the type-feature A feature request or enhancement label Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting review type-feature A feature request or enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants